Determining an undesired action of a vehicle

ABSTRACT

It is presented a method performed in a vehicle action determiner for determining an undesired action of a first vehicle and a resulting action. The method comprises: detecting that a first user is focusing on the first vehicle; acquiring brain activity data of the first user; determining a negative reaction of the first user based on the brain activity data; determining when the first vehicle is performing an undesired action based on the first user focusing on the first vehicle and the negative reaction of the first user; determining the resulting action taken by an autonomous vehicle, based on the first vehicle performing the undesired action; and triggering the autonomous vehicle to perform the resulting action.

This application is a 35 U.S.C. § 371 national phase filing ofInternational Application No. PCT/SE2017/051235, filed Dec. 8, 2017, thedisclosure of which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

The invention relates to a method, vehicle action determiners, acomputer program and a computer program product for determining anundesired action of a vehicle.

BACKGROUND

There is a great deal of research and development currently in the fieldof vehicles with autonomous propulsion capability. Applications rangefrom passenger transport such as bus and taxi services, to goodstransports performed by heavy-duty vehicles. In recent years, manycompanies have validated their autonomous heavy-duty vehicles in realroads. The Google driverless car has been operating in several areas inthe United States since 2012. In 2014, Google reported that their carshave completed a total of 1.1 million km in autonomous mode inCalifornia. Furthermore, aerial vehicles, such as drones, can beautonomous vehicles.

Machine learning is used in many autonomous vehicles to improve drivingcapability over time. The performance of such a machine learning modelbenefits from both quantity and quality of feedback. However, thequantity of feedback is always limited and any new sources of feedbackwould greatly benefit the performance of machine learning based driving.

SUMMARY

It is an object of embodiments presented herein to provide an improvedfeedback to autonomous vehicles based on vehicles performing undesiredactions.

According to a first aspect, it is presented a method for determining anundesired action of a first vehicle and a resulting action. The methodis performed in a vehicle action determiner and comprises the steps of:detecting user focus, by detecting that a first user is focusing on thefirst vehicle; acquiring brain activity data of the first user;determining a negative reaction of the first user based on the brainactivity data; determining when the first vehicle is performing anundesired action based on the first user focusing on the first vehicleand the negative reaction of the first user; determining the resultingaction to be taken by an autonomous vehicle, based on the first vehicleperforming the undesired action; and triggering the autonomous vehicleto perform the resulting action.

The steps of detecting user focus, acquiring brain activity data may berepeated for a second user. In such a case, the step of determining whenthe first vehicle is performing an undesired action is further based onthe second user focusing on the first vehicle and a negative reaction ofthe second user.

The method may further comprise the step of: acquiring controlparameters of the first vehicle. In such a case, the step of determiningwhen the first vehicle is performing an undesired action is furtherbased on the control parameters of the first vehicle.

The control parameters may comprise at least one parameter describing acurrent motion of the first vehicle.

The control parameters may comprise at least one parameter relating toroad characteristics of a road driven by the first vehicle.

The step of determining a resulting action may comprise consideringcontrol parameters of the first vehicle.

The step of detecting user focus may be based on gaze tracking.

In the step of acquiring brain activity data, the brain activity datamay be based on electroencephalography, EEG.

The step of detecting user focus may comprise only considering firstvehicles within a specific distance from the first user.

The step of determining when the first vehicle is performing anundesired action may further be based on a current geographic locationof the first vehicle.

The step of determining when the first vehicle is performing anundesired action may further be based on historic driving data of thefirst vehicle.

In the step of determining a resulting action, the resulting action maybe a corrective control action of the autonomous vehicle.

The method may further comprise the step of: providing, when the firstvehicle is not performing an undesired action, feedback to the firstuser, indicating that the first vehicle is not performing an undesiredaction.

According to a second aspect, it is presented a vehicle actiondeterminer for determining an undesired action of a first vehicle and aresulting action. The vehicle action determiner comprises: a processor;and a memory storing instructions that, when executed by the processor,cause the vehicle action determiner to: detect user focus, by detectingthat a first user is focusing on the first vehicle; acquire brainactivity data of the first user; determine a negative reaction of thefirst user based on the brain activity data; determine when the firstvehicle is performing an undesired action based on the first userfocusing on the first vehicle and the negative reaction of the firstuser; determine a resulting action to be taken by an autonomous vehicle,based on the first vehicle performing the undesired action; and triggerthe autonomous vehicle to perform the resulting action.

The instructions to detect user focus, acquire brain activity data maybe repeated for a second user. In such a case, the instructions todetermine when the first vehicle is performing an undesired action mayfurther be based on the second user focusing on the first vehicle and anegative reaction of the second user.

The vehicle action determiner may further comprise instructions that,when executed by the processor, cause the vehicle action determiner to:acquire control parameters of the first vehicle. In such a case, theinstructions to determine when the first vehicle is performing anundesired action is further based on the control parameters of the firstvehicle.

The control parameters may comprise at least one parameter describing acurrent motion of the first vehicle.

The control parameters may comprise at least one parameter relating toroad characteristics of a road driven by the first vehicle.

The instructions to determine a resulting action may compriseinstructions that, when executed by the processor, cause the vehicleaction determiner to consider control parameters of the first vehicle.

The instructions to detect may comprise instructions that, when executedby the processor, cause the vehicle action determiner to detect userfocus based on gaze tracking.

In the instructions to acquire brain activity data, the brain activitydata may be based on electroencephalography, EEG.

The instructions to detect user focus may comprise instructions that,when executed by the processor, cause the vehicle action determiner toonly consider first vehicles within a specific distance from the firstuser.

The instructions to determine when the first vehicle is performing anundesired action may comprise instructions that, when executed by theprocessor, cause the vehicle action determiner to determine that thefirst vehicle is performing an undesired action further based on acurrent geographic location of the first vehicle.

The instructions to determine when the first vehicle is performing anundesired action may comprise instructions that, when executed by theprocessor, cause the vehicle action determiner to determine when thefirst vehicle is performing an undesired action further based onhistoric driving data of the first vehicle.

In the instructions to determine a resulting action, the resultingaction may be a corrective control action of the autonomous vehicle.

The vehicle action determiner may further comprise instructions that,when executed by the processor, cause the vehicle action determiner toprovide, when the first vehicle is not performing an undesired action,feedback to the first user, indicating that the first vehicle is notperforming an undesired action.

According to a third aspect, it is presented a vehicle action determinercomprising: means for detecting user focus, by detecting that a firstuser is focusing on a first vehicle; means for acquiring brain activitydata of the first user; means for determining a negative reaction of thefirst user based on the brain activity data; means for determining whenthe first vehicle is performing an undesired action based on the firstuser focusing on the first vehicle and the negative reaction of thefirst user; means for determining the resulting action to be taken by anautonomous vehicle, based on the first vehicle performing the undesiredaction; and means for triggering the autonomous vehicle to perform theresulting action.

According to a fourth aspect, it is presented a computer program fordetermining an undesired action of a first vehicle and a resultingaction. The computer program comprises computer program code which, whenrun on a vehicle action determiner causes the vehicle action determinerto: detect user focus, by detecting that a first user is focusing on thefirst vehicle; acquire brain activity data of the first user; determinea negative reaction of the first user based on the brain activity data;determine when the first vehicle is performing an undesired action basedon the first user focusing on the first vehicle and the negativereaction of the first user; determine the resulting action to be takenby an autonomous vehicle, based on the first vehicle performing theundesired action; and trigger the autonomous vehicle to perform theresulting action.

According to a fifth aspect, it is presented a computer program productcomprising a computer program according to the fourth aspect and acomputer readable means on which the computer program is stored.

Generally, all terms used in the claims are to be interpreted accordingto their ordinary meaning in the technical field, unless explicitlydefined otherwise herein. All references to “a/an/the element,apparatus, component, means, step, etc.” are to be interpreted openly asreferring to at least one instance of the element, apparatus, component,means, step, etc., unless explicitly stated otherwise. The steps of anymethod disclosed herein do not have to be performed in the exact orderdisclosed, unless explicitly stated.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is now described, by way of example, with reference to theaccompanying drawings, in which:

FIGS. 1A-B are schematic drawings illustrating an environment in whichembodiments presented herein can be applied;

FIGS. 2A-B are schematic drawings illustrating brain activity sensingaccording to two embodiments which can be applied in the environment ofFIGS. 1A-B;

FIG. 3 is a schematic diagram illustrating components of a vehicleaction determiner that can be used in the environment of FIGS. 1A-B;

FIGS. 4A-D are schematic diagrams illustrating embodiments of how thevehicle action determiner of FIG. 3 can be implemented;

FIGS. 5A-C are flow charts illustrating methods for determining anundesired action of a first vehicle and a resulting action, the methodbeing performed in the vehicle action determiner of FIG. 3 ;

FIG. 6 is a schematic diagram showing functional modules of the vehicleaction determiner of FIG. 3 according to one embodiment; and

FIG. 7 shows one example of a computer program product comprisingcomputer readable means.

DETAILED DESCRIPTION

The invention will now be described more fully hereinafter withreference to the accompanying drawings, in which certain embodiments ofthe invention are shown. This invention may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided byway of example so that this disclosure will be thorough and complete,and will fully convey the scope of the invention to those skilled in theart. Like numbers refer to like elements throughout the description.

Embodiments presented herein are based on the realisation that thereaction of users can be utilised to provide feedback on vehicleactions. Specifically, the feedback is based on detecting brain activitydata from a user to find out when the user has a negative reaction. Thisis coupled to an vehicle being the focus of the user to thereby providenegative feedback for an autonomous vehicle to learn from thatsituation. Since the feedback is based on brain sensing, the user doesnot need to provide any feedback using a user interface, whereby thesystem can work without user input to a mass large amounts of veryvaluable feedback data for the performance of autonomous vehicles.

FIGS. 1A-B are schematic drawings illustrating an environment in whichembodiments presented herein can be applied. There are here fourvehicles 2 a-d (of which three are shown in FIG. 1A). At least the firstvehicle 2 a and the fourth vehicle 2 d are autonomous vehicles, beingcapable of autonomous propulsion for at least part of the time. Thesecond vehicle 2 b and the third vehicle 2 c can also be autonomousvehicles or can be humanly controlled vehicles. For instance, theautonomous vehicles can be a car with autonomous driving capability oran aerial vehicle (e.g. a drone) with autonomous flight capability. Theautonomous vehicles comprise one or more cameras and other sensors whichare used to analyse the environment of the vehicle and control thevehicle accordingly. The control of the vehicle can be based on amachine learning model. The machine learning model can be improved byfeeding appropriate feedback to the model, including resulting actions.The more feedback that is available to train the machine learning model,the better the performance. In other words, each instance of a specificset of features as input (e.g. one or more of the control parametersmentioned below) and a known output, such as a resulting actiondetermined from an undesired action, can be used to train the machinelearning model further. When such combinations of inputs and outputs amass in large numbers, as is made possible by embodiments presentedherein since no user interaction is needed, this results in continuousimprovement of the machine learning model. The vehicles 2 a-c can alsocomprise additional sensors for acquiring control parameters which arerelevant for controlling the vehicle such as audio, vehicle motion,vehicle speed, road conditions, aerial route conditions etc. Thevehicles 2 a-c can also acquire external control parameters such astraffic conditions, road accidents/construction zones, weatherconditions, etc. by communicating with neighbouring vehicles and/or aserver. The vehicles 2 a-c contain a control unit which defines how thevehicle should be operated in autonomous mode, e.g. based on the machinelearning model.

In the scenarios herein, the local regulations stipulate right handtraffic. In the scenario of FIG. 1A, a user 5 is travelling in the firstautonomous vehicle 2 a. A second autonomous vehicle 2 b is travelling inthe opposite direction. However, a third vehicle 2C is overtaking thesecond autonomous vehicle 2 b, in a way that is dangerous for the firstautonomous vehicle 2 a and thus the user 5. The third vehicle 2 c can bean autonomous vehicle or a traditional, human controlled vehicle.According to embodiments presented herein, the user 5 will notice theaction of the autonomous vehicle 2 c and a brain activity acquirer willcapture data from the brain of the user 5 from which it can be deducedthat the user 5 has a negative reaction. A negative reaction is anegative emotional reaction of the user than can be detected. When humanobserve other humans committing errors, their brains show a specificactivation pattern in response to these errors, which can e.g. reflectfeelings such as fear, anger, disgust, etc. This will be used to allowthe system to learn from the situation and will reduce the risk ofautonomous vehicles acting in the same way as the third vehicle 2 c inthis scenario.

In FIG. 1B, a scenario is shown where the user 5 of a fourth autonomousvehicle 2 d also notices the undesired action of the third vehicle 2 c,giving more weight to the situation.

FIGS. 2A-B are schematic drawings illustrating brain activity sensingaccording to two embodiments which can be applied in the environment ofFIGS. 1A-B.

In FIG. 2A, the user 5 is shown sitting in a seat of an autonomousvehicle, e.g. the first autonomous vehicle 2 a of FIGS. 1A-B. The useris wearing a brain activity acquirer 7 in the form of a head mountedsensor arrangement. A camera 4 is directed towards the eyes of the userto allow gaze tracking, as known in the art per se. Using gaze tracking,it can be determined where the user 5 is looking, e.g. at a specificvehicle.

Looking now to the brain sensor device 7. Brain sensor devices, alsoknown as brain computer interfaces (BCI) are devices that are capable ofmeasuring the brain activity of a user, typically throughElectroencephalography (EEG). These brain sensor devices 7 can be in theform of an externally mounted head mounted device, an outer-ear orinner-ear mounted device which may be considered wearable brain sensordevices 7. A brain sensor device 7 could also be an embedded device inthe user head.

The brain sensor device 7 can be used to detect various aspects of userbrain activity. One such activity is to detect when a user experiences anegative reaction. Compared to other visionary uses of brain activitydetection, the negative reaction detection is much more straightforwardand feasible using data provided by the brain sensor devices 7 availabletoday. For instance, it is disclosed in Welke et al: “Brain ResponsesDuring Robot-Error Observation”, available athttps://arxiv.org/ftp/arxiv/papers/1708/1708.01465.pdf at the time offiling this application, how EEG is used to differentiate when a personobserves a correct vs erroneous action of a robot.

In FIG. 2B, the brain sensor device 7 is implemented as part of aheadrest of the seat of the user 5. In this way, no device needs to beworn by the user.

FIG. 3 is a schematic diagram illustrating components of a vehicleaction determiner 1 that can be used in the environment of FIGS. 1A-B.When the vehicle action determiner is implemented as part of a hostdevice (see FIGS. 4A-C and description below), one or more of thesecomponents can be shared with the host device. A processor 60 isprovided using any combination of one or more of a suitable centralprocessing unit (CPU), multiprocessor, microcontroller, digital signalprocessor (DSP), application specific integrated circuit etc., capableof executing software instructions 67 stored in a memory 64, which canthus be a computer program product. The processor 60 can be configuredto execute the method described with reference to FIGS. 5A-C below.

The memory 64 can be any combination of random access memory (RAM) andread only memory (ROM). The memory 64 also comprises persistent, i.e.non transitory, storage, which, for example, can be any single one orcombination of magnetic memory, optical memory, solid-state memory oreven remotely mounted memory.

A data memory 66 is also provided for reading and/or storing data duringexecution of software instructions in the processor 60. The data memory66 can be any combination of random access memory (RAM) and read onlymemory (ROM).

The vehicle action determiner 1 further comprises an I/O interface 62for communicating with other external entities. Optionally, the I/Ointerface 62 further includes a user interface.

Other components of the vehicle action determiner 1 are omitted in ordernot to obscure the concepts presented herein.

The vehicle action determiner 1 can connect to a brain sensor device 7,which can be any of the brain sensor devices 7 of FIGS. 2A-B,respectively. The vehicle action determiner 1 can also connect to aserver 8.

Moreover, the vehicle action determiner 1 can connect to an autonomousvehicle 2, which may be an autonomous vehicle in which the user istravelling. The vehicle action determiner 1 can also connect to a camera4 to allow gaze tracking of the user.

FIGS. 4A-D are schematic diagrams illustrating embodiments of how thevehicle action determiner 1 of FIG. 3 can be implemented. The vehicleaction determiner 1 can be implemented as a stand-alone device or it canform part of a host device, as shown in FIGS. 4A-C.

In FIG. 4A, the vehicle action determiner 1 is shown as implemented inthe autonomous vehicle 2. The autonomous vehicle 2 is thus the hostdevice for the vehicle action determiner 1. In this case, the autonomousvehicle can be, but does not need to be, the autonomous vehicle in whichthe user is travelling at the moment.

In FIG. 4B, the vehicle action determiner 1 is shown as implemented inthe brain sensor device 7. The brain sensor device 7 is thus the hostdevice for the vehicle action determiner 1. In this embodiment, thebrain sensor device acquires brain data of a user being anywhere wherethe focus of the user can be detected. For instance, the user can besituated by the side of the road with external cameras used to detectuser focus. Alternatively, the brain sensor device also comprises acamera to detect eye gaze of the user. The brain sensor device 7 thenuses the brain data of the user and eye gaze of the user to determinewhether an undesired of an observed vehicle occurs.

In FIG. 4C, the vehicle action determiner 1 is shown as implemented inthe server 8. The server 8 is thus the host device for the vehicleaction determiner 1. In this embodiment, the server 8 can be connectedto a brain activity acquirer for acquiring brain data of a user beinganywhere where the focus of the user can be detected. For instance, theuser can be situated by the side of the road with external cameras usedto detect user focus. The server then uses the brain data of the userand eye gaze of the user to determine whether an undesired of anobserved vehicle occurs.

In FIG. 4D, the vehicle action determiner 1 is shown implemented as astand-alone device. In this embodiment, there is no host device for thevehicle action determiner 1.

FIGS. 5A-C are flow charts illustrating methods for determining anundesired action of a first vehicle and a resulting action for anautonomous vehicle, the method being performed in the vehicle actiondeterminer of FIG. 3 . The user is travelling in the autonomous vehicle.The first vehicle can be the same as the autonomous vehicle in which theuser is travelling. Alternatively, the first vehicle is external to theautonomous vehicle in which the user is travelling. The first vehiclecan be an autonomous vehicle or a traditional, human controlled vehicle.

In a detect user focus step 40, the vehicle action determiner detectsuser focus, by detecting that a first user is focusing on a firstvehicle. The term first here is only used to allow reference to thefirst vehicle in text below. The term first does not imply any order orpriority in any way. The user focus on the first vehicle can e.g. bebased on gaze tracking (using the cameras 4 shown in FIGS. 2A-B). It canalso be determined when the user is focusing on its own autonomousvehicle, i.e. the autonomous vehicle in which the user is travelling.

Optionally, only vehicles within a specific distance from the first userare considered to be potential first vehicles, as vehicles far away maybe difficult for the user to properly analyse.

Another camera of the vehicle where the user is located (e.g. adashboard camera), a camera on a user device close to the user (e.g. ahead mounted display used by the autonomous vehicle). Alternatively oradditionally, a camera external to the vehicle can be utilised. The usereye gaze is identified in the images acquired by the cameras by knowingthe relative position between the camera and the current user location,which may be known (e.g. user is sitting in the seat and has height X)or may be detected in an online manner with communication with theserver 8 (e.g. another camera facing inside the car detects where theuser head and eyes are located and computes the transform of positionand direction between the user eyes location and the camera).

In an acquire brain activity data step 42, the vehicle action determineracquires brain activity data of the first user. This can be performedusing the brain sensor device 7 shown in FIGS. 2A-B. For instance, thebrain activity data can be based on electroencephalography (EEG).

In a determine negative reaction step 44, the vehicle action determinerdetermines a negative reaction of the first user based on the brainactivity data. As mentioned above, compared to other visionary uses ofbrain activity detection, the negative reaction detection is much morestraightforward and feasible using data provided by the brain sensordevices available today. The determination of negative reaction can e.g.be based on machine learning.

In a conditional undesired action step 46, the vehicle action determinerdetermines whether the first vehicle is performing an undesired actionbased on the first user focusing on the first vehicle and the negativereaction of the first user. In other words, when the user is focusing ona particular vehicle and brain waves indicate that the user is having anegative reaction, this is interpreted as that the first vehicleperforming an undesired action.

This determination can optionally be based on a current geographiclocation of the first vehicle (or the autonomous vehicle). Acceptabledriving behaviour can vary significantly between different countries andthis can be considered when determining whether the action is undesiredor not.

Furthermore, the determination is optionally based on historic drivingdata of the first vehicle. For instance, if historic data indicate thatmany similar actions have not been associated with any undesiredbehaviours for tens of thousands of kilometres, this shifts the weightof the action to not be considered to be an undesired action.

When an undesired action occurs, the method proceeds to a determineresulting action step 48. Otherwise, the method ends.

In the determine resulting action step 48, the vehicle action determinerdetermines the resulting action to be taken by the autonomous vehicle,based on the first vehicle performing the undesired action. Theresulting action can be used to train the machine learning model for theautonomous vehicle, where the inputs of the model contain one or morecontrol parameters and the output contains the resulting action or aconsequence thereof. Such training of machine learning models can beused as part of shared learning, expanding to improve the machinelearning models for several vehicles. For instance, all autonomousvehicles whose owners give the permission to learn from other vehiclescan learn and avoid an undesired action in a similar context in thefuture using such shared learning. Such shared learning can becommunicated via a server or directly from the vehicle action determinerto the autonomous vehicle in question.

The resulting action can be a corrective control action of theautonomous vehicle. The corrective control action can be a real-timecontrol action which influences behaviour of the autonomous vehicle inthe current situation, which has been determined to be undesired. Suchreal-time corrective control action can e.g. be applied when the firstvehicle (i.e. the observed vehicle) is also the vehicle which istrained. Alternatively or additionally, the corrective control action isstored in memory of the autonomous vehicle such that a correspondingundesired action can be avoided in a similar context in the future.

In one embodiment, the resulting action is that the autonomous vehiclerequests additional information from a human driver to learn for asimilar situation in the future. The human driver may be located insidethe autonomous vehicle or remotely, to provide input to the machinelearning model in the situation of the undesired action. The autonomousvehicle supplies data corresponding to the situation of the undesiredaction which was previously recorded. This allows the autonomous vehicleto learn how to correct such action.

In one embodiment, the resulting action is that the autonomous vehicleis mandated to receive additional training, e.g. from a simulatorsimulating the situation where the undesired action occurs. Optionally,this training can be triggered only when the number of such undesiredactions exceeds a certain threshold. This resulting action reduces therisk that the autonomous vehicle performing the undesired action willperform such an action when a similar situation occurs in the future.

In one embodiment, the resulting action is that the autonomous vehiclerequests additional information from a human driver for assistance whena similar situation as the undesired action occurs in the future. Theautonomous vehicle will then ask a human driver (in the autonomousvehicle or remotely) to take over and drive the vehicle so that theautonomous vehicle can learn from the driver.

In one embodiment, the resulting action is that the autonomous vehicleis caused to perform an alternative action to the undesired action whenthe autonomous vehicle re-encounters the situation of the undesiredaction in the future. As an example, the autonomous vehicle may have twopotential actions which can be performed in the situation, as action Aand action B, and the autonomous vehicle will use action B if action Ahas been reported as undesired.

In one embodiment, the resulting action is that the autonomous vehicleaccumulates the undesired action occurrences, and if the number ofundesired action occurrences exceeds a given threshold, the autonomousvehicle decides to act upon the situation, either by requestingadditional information, or by altering the action to be performed in thesituation of the undesired action. Similarly, when the occurrences ofundesired action for the action in the same situation performed by a setof autonomous vehicles accumulate to undesired action occurrencesexceeding a predetermined threshold, the remaining autonomous vehiclesof the same brand, owner or the autonomous vehicles in a certaingeographic location also modify their action in that situation.

In one embodiment, the resulting action is that the autonomous vehicleperforms at least one of the above mentioned resulting actions, weightedon the performance goals of the autonomous vehicle. As an example, itmay be determined to be undesired that a vehicle has not accelerated tocross an amber traffic light before turning red. However, suchacceleration would have cost the vehicle a larger energy expenditurewhich the vehicle has been set to optimise by the owner or by themanufacturer, or exceed a maximum acceleration limitation imposed on thevehicle by the same. Similarly, the autonomous vehicle may also want tooptimise its physical conditions, such as motor condition, breaks, etc.Hence a weighted function which considers all these factors may be usedwhen deciding if a modification of the resulting action should beperformed.

When the resulting action results in the autonomous vehicle learningfrom the experience, the feedback of the resulting action can be sharedto more autonomous vehicles to allow their respective machine learningmodels to be trained.

In a trigger resulting action step 49, the vehicle action determinertriggers the autonomous vehicle to perform the resulting action.

By using brain activity data, a recognition of an undesired action isachieved without the need for explicit user input. This can be used foran autonomous vehicle to learn from the undesired action. The feedbackfrom the user can be readily correlated to the current activity beingperformed by the autonomous vehicle.

Embodiments presented herein allow for feedback to several autonomousvehicles and not just for the autonomous vehicle in which the user istravelling. The embodiments enable a quick and robust determination ofany undesired action of vehicles. Since the feedback can occur withoutuser interaction, large amounts of feedback data can be amassed togreatly improve the performance of the autonomous vehicles.

Looking now to FIG. 5B, only new or modified steps compared to the stepsof FIG. 5A will be described.

In an optional acquire control parameters step 45, the vehicle actiondeterminer acquires control parameters of the first vehicle.

The control parameters comprise at least one parameter describing acurrent motion of the first vehicle. Current motion is here to beinterpreted as any one or more of velocity, acceleration, direction,etc.

Alternatively or additionally, the control parameters comprise at leastone parameter relating to road characteristics of a road driven by thefirst vehicle. Road characteristics can e.g. not only relate to thenature of the road, such as curvature, incline, number of lanes, width,speed limit, traffic regulations, limits on acceleration/decelerationalong the driving direction, lateral acceleration etc., but also to atemporary condition of the road, such as traffic conditions, roadclosures, amount of snow, ice or leaves and/or weather conditions suchas temperature, rain, snow, sunshine and sun angle, etc. Furthermore,activity identification of the first vehicle can form part of thecontrol parameters, such as driving straight, overtaking another vehicleand returns to original lane, driving through an amber traffic light,etc. Optionally, the control parameters also comprise information aboutthe surroundings, e.g. other vehicles, pedestrians, bicyclists, etc.,which is data that can be collected by the autonomous vehicle.

When the control parameters are available, the conditional undesiredaction step 46 can also be based on any one or more of the controlparameters of the first vehicle.

When the control parameters are available, the determine resultingaction step 48 can also be based on one or more of the controlparameters of the first vehicle.

In an optional provide feedback step 54, when there is no undesiredaction determined in step 46, the vehicle action determiner providesfeedback to the first user, indicating that the first vehicle is notperforming an undesired action.

Looking now to FIG. 5C, the method illustrated here is similar to themethod of FIG. 5A, but based on at least two users.

Here, steps 40, 42 and 44 are performed in two separate paths, one for afirst user and one for a second user. In here, both users focus on thesame first vehicle. In each instance of step 44, there is a binaryclassification of whether there is a negative reaction or not. Thenegative reaction or not is determined in each instance of step 44 asdescribed in more detail above.

In the conditional undesired action step 46, the negative reaction anduser focus from both users are used to determine whether the firstvehicle is performing an undesired action.

Hence the brain activity data of at least two users are analysed,resulting in binary classifications of negative reaction or not in step44. These binary classifications are combined in the step 46, in thedetermination of whether the first vehicle is performing an undesiredaction or not. The binary classifications can be combined in severaldifferent ways.

In one embodiment, the determination is based on an average of thebinary classifications obtained from all users currently focusing on thefirst vehicle.

In one embodiment, the determination is based on the average of theclassifications obtained from all users which are closer than a certaindistance from the first vehicle and are currently focusing on the firstvehicle.

In one embodiment, the determination is based on the average of theclassifications obtained from users currently focusing on the firstvehicles, but only those users that are locals in the current geographicarea are considered. This considers cultural (e.g., Europeans vs Asiansvs Americans, South Europeans vs North Europeans) and geographicinterpretations of an undesired action, where a certain behaviour may beconsidered undesired in one country, but not in another.

In one embodiment, the determination is based on the average of theclassifications obtained from all users currently focusing on the firstvehicle, and also based on previous classifications of the same actionwhich have been stored. As an example, the first vehicle might be sharedamong various users, where the various users have previously classifiedthe same action which is currently being performed by the first vehicle,and the stored result is used for the determination of the undesiredbehaviour.

In one embodiment, the determination is based on the average of theclassifications obtained from all users currently focusing on the firstvehicle, and also based on previous knowledge from the first vehicleabout the action. The previous knowledge about the action can e.g. beobtained by previous training (real or simulation of human driver),and/or the driver's manual. As an example, the first vehicle may have astrong belief that such action is not undesired due to previousknowledge.

In one embodiment, the determination is based on the average of theclassifications obtained from all users focusing on the first vehicle,as well as the road regulations which can be retrieved from the drivingcode regulations.

Hence, given the EEG recordings from at least two users, the currentaction being performed by multiple vehicles is classified as undesired.As an example, if multiple vehicles are involved in an undesired action(e.g. users which focus on both vehicle A and vehicle B report undesiredactions), the undesired action may not be trivially assigned to aspecific vehicle. Hence, the classification from multiple users must beconsidered. As an example, consider that vehicle A and B are involved inan undesired action. The classifications from users which focus onvehicle A or vehicle B are considered, for example, to distinguish if alarger number of undesired classifications allocated to vehicle A thanvehicle B, or vice versa. This embodiment could be implemented on theserver (8 of FIG. 3 and FIG. 4C).

After step 46, the method performs in an analogous way to what isillustrated in FIGS. 5A and 5B.

FIG. 6 is a schematic diagram showing functional modules of the vehicleaction determiner 1 of FIG. 3 according to one embodiment. The modulesare implemented using software instructions such as a computer programexecuting in the vehicle action determiner 1. Alternatively oradditionally, the modules are implemented using hardware, such as anyone or more of an ASIC (Application Specific Integrated Circuit), anFPGA (Field Programmable Gate Array), or discrete logical circuits. Themodules correspond to the steps in the methods illustrated in FIGS.5A-5C.

A user focus detector 70 corresponds to step 40. A brain activity dataacquirer 72 corresponds to step 42. A negative reaction determiner 74corresponds to step 44. A control data acquirer m corresponds to step45. An undesired action determiner 76 corresponds to step 46. Aresulting action determiner 78 corresponds to step 48. An action trigger79 corresponds to step 49. A feedback provider 84 corresponds to step54.

FIG. 7 shows one example of a computer program product 90 comprisingcomputer readable means. On this computer readable means, a computerprogram 91 can be stored, which computer program can cause a processorto execute a method according to embodiments described herein. In thisexample, the computer program product is an optical disc, such as a CD(compact disc) or a DVD (digital versatile disc) or a Blu-Ray disc. Asexplained above, the computer program product could also be embodied ina memory of a device, such as the computer program product 67 of FIG. 3. While the computer program 91 is here schematically shown as a trackon the depicted optical disk, the computer program can be stored in anyway which is suitable for the computer program product, such as aremovable solid state memory, e.g. a Universal Serial Bus (USB) drive.

The invention has mainly been described above with reference to a fewembodiments. However, as is readily appreciated by a person skilled inthe art, other embodiments than the ones disclosed above are equallypossible within the scope of the invention, as defined by the appendedpatent claims.

The invention claimed is:
 1. A method for determining an undesiredaction of a first vehicle and a resulting action, the method beingperformed in a vehicle action determiner and comprising the steps of:detecting user focus, by detecting that a first user is focusing on thefirst vehicle; determining a distance to the first vehicle; acquiringbrain activity data of the first user, wherein the brain activity datais based on an electroencephalography measurement; determining anegative reaction of the first user based on the brain activity data;determining when the first vehicle is performing an undesired actionbased on the first user focusing on the first vehicle and the negativereaction of the first user; determining the resulting action to be takenby an autonomous vehicle, based on the first vehicle performing theundesired action and the distance to the first vehicle being less than apredefined distance; and triggering the autonomous vehicle to performthe resulting action.
 2. The method according to claim 1, wherein thesteps of detecting user focus, acquiring brain activity data arerepeated for a second user and wherein the step of determining when thefirst vehicle is performing an undesired action is further based on thesecond user focusing on the first vehicle and a negative reaction of thesecond user.
 3. The method according to claim 1, further comprising thestep of: acquiring control parameters of the first vehicle; and whereinthe step of determining when the first vehicle is performing anundesired action is further based on the control parameters of the firstvehicle.
 4. The method according to claim 3, wherein the controlparameters comprise at least one parameter describing a current motionof the first vehicle.
 5. The method according to claim 3, wherein thecontrol parameters comprise at least one parameter relating to roadcharacteristics of a road driven by the first vehicle.
 6. The methodaccording to claim 3, wherein the step of determining a resulting actioncomprises considering control parameters of the first vehicle.
 7. Themethod according to claim 1, wherein the step of detecting user focus isbased on gaze tracking.
 8. A vehicle action determiner for determiningan undesired action of a first vehicle and a resulting action, thevehicle action determiner comprising: a processor; and a memory storinginstructions that, when executed by the processor, cause the vehicleaction determiner to: detect user focus, by detecting that a first useris focusing on the first vehicle; determining a distance to the firstvehicle; acquire brain activity data of the first user, wherein thebrain activity data is based on an electroencephalography measurement;determine a negative reaction of the first user based on the brainactivity data; determine when the first vehicle is performing anundesired action based on the first user focusing on the first vehicleand the negative reaction of the first user; determine the resultingaction to be taken by an autonomous vehicle, based on the first vehicleperforming the undesired action and the distance to the first vehiclebeing less than a predefined distance; and trigger the autonomousvehicle to perform the resulting action.
 9. The vehicle actiondeterminer according to claim 8, wherein the instructions to detect userfocus, and to acquire brain activity data are repeated for a second userand wherein the instructions to determine when the first vehicle isperforming an undesired action is further based on the second userfocusing on the first vehicle and a negative reaction of the seconduser.
 10. The vehicle action determiner according to claim 8, furthercomprising instructions that, when executed by the processor, cause thevehicle action determiner to: acquire control parameters of the firstvehicle; and wherein the instructions to determine when the firstvehicle is performing an undesired action is further based on thecontrol parameters of the first vehicle.
 11. The vehicle actiondeterminer according to claim 10, wherein the control parameterscomprise at least one parameter describing a current motion of the firstvehicle.
 12. The vehicle action determiner according to claim 10,wherein the control parameters comprise at least one parameter relatingto road characteristics of a road driven by the first vehicle.
 13. Thevehicle action determiner according to claim 10, wherein theinstructions to determine a resulting action comprise instructions that,when executed by the processor, cause the vehicle action determiner toconsider control parameters of the first vehicle.
 14. The vehicle actiondeterminer according to claim 8 wherein the instructions to detectcomprise instructions that, when executed by the processor, cause thevehicle action determiner to detect user focus based on gaze tracking.15. The vehicle action determiner according to claim 8, wherein theinstructions to determine when the first vehicle is performing anundesired action comprise instructions that, when executed by theprocessor, cause the vehicle action determiner to determine that thefirst vehicle is performing an undesired action further based on acurrent geographic location of the first vehicle.
 16. The vehicle actiondeterminer according to claim 8, wherein the instructions to determinewhen the first vehicle is performing an undesired action compriseinstructions that, when executed by the processor, cause the vehicleaction determiner to determine when the first vehicle is performing anundesired action further based on historic driving data of the firstvehicle.
 17. The vehicle action determiner according to claim 8, whereinin the instructions to determine a resulting action, the resultingaction is a corrective control action of the autonomous vehicle.
 18. Thevehicle action determiner according to claim 8, further comprisinginstructions that, when executed by the processor, cause the vehicleaction determiner to provide, when the first vehicle is not performingan undesired action, feedback to the first user, indicating that thefirst vehicle is not performing an undesired action.
 19. Anon-transitory computer-readable medium comprising a computer programfor determining an undesired action of a first vehicle and a resultingaction that when executed by a processor of a vehicle action determinercause, the computer program comprising computer program code which, whenrun on a vehicle action determiner causes the vehicle action determinerto: detect user focus, by detecting that a first user is focusing on thefirst vehicle; determining a distance to the first vehicle; acquirebrain activity data of the first user, wherein the brain activity datais based on an electroencephalography measurement; determine a negativereaction of the first user based on the brain activity data; anddetermine when the first vehicle is performing an undesired action basedon the first user focusing on the first vehicle and the negativereaction of the first user and the distance to the first vehicle beingless than a predefined distance.